Instrument reliability and QC frequency: a cautionary tale.

I magine this announcement: “Accuration Instruments Inc. is proud to announce the release of the Reliabulator 2/500, a new hallmark in the reliability of clinical diagnostic laboratory automation.” Your laboratory will be one of the fi rst installations of the new Reliabulator 2/500. The manufacturer claims the “2” in the instrument name stands for an expected two years between undetected grave malfunctions, and the “500” means it will process 500 patient specimens per day. One of the things you need to decide on is a suitable qualitycontrol strategy for the Reliabulator. If it only fails every two years, does that mean no quality control (QC) is necessary the fi rst year? That would certainly help the budget. After a bit more consideration, you realize that an expected malfunction rate of once every two years is not the same thing as a malfunction every two years starting after installation. The instrument could have an expected malfunction rate of once every two years and still malfunction the day after it is validated. An expected malfunction rate of once every two years means that the malfunction could occur at any time; but over the long term, there will be an average of one malfunction for every two years of service. Consequently, no QC during the fi rst year is not much of an option. What about doing QC once a month? What is the worstcase scenario? If QC is done on the fi rst of the month and the system malfunctions on the next sample, that would be the worst case. If the malfunction was not detected until the next QC, that would mean a whole month’s worth of compromised patient samples — call it 30 days of 500 patients per day: 15,000 compromised specimens! But how likely is it that a malfunction will occur on the specimen right after a QC event? The answer is: “Not very likely.” What about the best-case scenario? The system might malfunction after the last patient specimen of the month but before the QC specimen next month. The malfunction is detected, and no patient specimens are affected. How likely is that to happen? The answer is “not very likely” but about the same probability of the system malfunctioning right after QC. Given that the greatest risk to the laboratory will occur if a malfunction happens on the fi rst of the month just after the QC event, and there is no risk to the laboratory if a malfunction happens at the end of the month after the last patient specimen (but before next month’s QC event), then the middle of the month can be used to estimate the expected number of patient specimens compromised by a malfunction — assuming that a malfunction is equally likely to occur on any day of the month. It turns out that half the number of patient specimens between QC events is the expected number of specimens that are compromised by a malfunction.1 If the malfunction is grave enough, half the number of patients between QC events will contain an unacceptable amount of error. In the case under consideration, 15 days of 500 patients per day would result in 7,500 compromised patient specimens, every two years, or 3,750 patients per year. Looking at a strategy of performing QC once a week, results in 3.5 days of 500 patient specimens per day that are compromised in the event of a malfunction — 1,750 every two years, an expected 875 per year. Evaluating a QC specimen every morning results in 0.5 days of 500 patient specimens per day that are compromised in the event of a malfunction — 250 every two years, an expected 125 per year, or about one every three days. This may be manageable, but it requires doing QC every day, even on an instrument as reliable as the hypothetical Reliabulator 2/500. Unfortunately, there is another consideration: although the expected rate of compromised patient results is one result every three days, they actually occur as 250 consecutive compromised results at one point in time, every two years on average — meaning a major headache when it happens. The reliability rate can be computed as the number of uncompromised patient specimens times 100/number of specimens. For a test system with periodic malfunctions, it is directly related to the frequency of QC events. For a QC frequency of once per month over a twoyear period, the reliability rate is calculated as 100* (365,000-7,500)/365,000 or 97.45% — or sigma value of 3.54.2